Tools and Methods to Analyze Multimodal Data in Collaborative Design Ideation

Collaborative design ideation is typically characterized by informal acts of sketching, annotation, and discussion. Designers have always used the pencil-and-paper medium for this activity, partly because of the flexibility of the medium, and partly because the ambiguous and ill-defined nature of conceptual design cannot easily be supported by computers. However, recent computational tools for conceptual design have leveraged the availability of hand-held computing devices for creating and sharing ideas. In order to provide computer support for collaborative ideation in a way that augments traditional media rather than imitates it, it is necessary to study the affordances made available by digital media for this process, and to study designers\u27 cognitive and collaborative processes when using such media. In this thesis, we present tools and methods to help make sense of unstructured verbal and sketch data generated during collaborative design, with a view to better understand these collaborative and cognitive processes. This thesis has three main contributions

ConceptScope: Organizing and Visualizing Knowledge in Documents based on Domain Ontology

Current text visualization techniques typically provide overviews of document content and structure using intrinsic properties such as term frequencies, co-occurrences, and sentence structures. Such visualizations lack conceptual overviews incorporating domain-relevant knowledge, needed when examining documents such as research articles or technical reports. To address this shortcoming, we present ConceptScope, a technique that utilizes a domain ontology to represent the conceptual relationships in a document in the form of a Bubble Treemap visualization. Multiple coordinated views of document structure and concept hierarchy with text overviews further aid document analysis. ConceptScope facilitates exploration and comparison of single and multiple documents respectively. We demonstrate ConceptScope by visualizing research articles and transcripts of technical presentations in computer science. In a comparative study with DocuBurst, a popular document visualization tool, ConceptScope was found to be more informative in exploring and comparing domain-specific documents, but less so when it came to documents that spanned multiple disciplines.Comment: 19 pages, 5 figure

ConceptEVA: Concept-Based Interactive Exploration and Customization of Document Summaries

With the most advanced natural language processing and artificial intelligence approaches, effective summarization of long and multi-topic documents -- such as academic papers -- for readers from different domains still remains a challenge. To address this, we introduce ConceptEVA, a mixed-initiative approach to generate, evaluate, and customize summaries for long and multi-topic documents. ConceptEVA incorporates a custom multi-task longformer encoder decoder to summarize longer documents. Interactive visualizations of document concepts as a network reflecting both semantic relatedness and co-occurrence help users focus on concepts of interest. The user can select these concepts and automatically update the summary to emphasize them. We present two iterations of ConceptEVA evaluated through an expert review and a within-subjects study. We find that participants' satisfaction with customized summaries through ConceptEVA is higher than their own manually-generated summary, while incorporating critique into the summaries proved challenging. Based on our findings, we make recommendations for designing summarization systems incorporating mixed-initiative interactions.Comment: 16 pages, 7 figure